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Abstract 

School climate has been acknowledged as a construct impacting important aspects 
of educational outcomes, such as student achievement, school effectiveness, and school 
completion. The Organizational Climate Index was an instrument developed to measure 
school climate (Hoy, 2001). This study evaluated this instrument by using Rasch 
measurement. The sample consisted of 1 10 teachers, in four schools. Results indicated 
that mostly items were working well, with a high reliability of .96. The only two items of 
concern referred to parent involvement in the school. This may be explained by the low 
community interaction in sample schools. 
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Using Rasch Measurement to Evaluate the Organizational Climate Index 

Educational literature has identified school climate as a construct impacting 
important aspects of educational outcomes, such as student achievement, school 
effectiveness (Pashiardis, 2000), and school completion (Worrel, 2000). Climate has 
often been referred to as the ‘personality’ or atmosphere of the school (Hoy, Hoffman, 
Sabo, and Bliss, 1996). It describes internal attributes shaped by the relationships among 
principal, teachers, students, and community. It also defines those relationships. 

Climates can be open or closed. In open climates, the relationships among 
teachers and principals are open and supportive; in closed climates, they are “guarded, 
suspicious, controlling, restrictive, distant, and disengaged” (Hoy et al., 1996 p.42). 
Climate is also described as positive, when the teaching and learning are emphasized, 
rewarded, built upon collegiality and collaboration (Pashiardis, 2000). According to their 
climates, schools can also be described as caring and supportive or hostile and 
nonsupportive (Worrel, 2000). In a review of the climate literature, Pashiardis (2000) 
presents the following characteristics of effective schools, principal is an instructional 
leader; rules, policies, procedures are clear, well understood, and implemented by staff; 
school presents a common plan, shared by community and staff; teachers individualize 
instruction to reach all students; there is communication and collaboration; there is high 
expectation and enthusiasm; and an atmosphere of professionalism is present as well as 
press for academic achievement. 

School climate has been assessed through a diversity of methods and study 
participants. For example, Dinham, Cairney, Craigie and Wilson (1995) used a 
combination of survey questionnaires, followed by in-depth case study, using interviews, 
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focus groups, and document analysis to assess the relationships among school climate, 
leadership, and decision making in three secondary schools. They gathered information 
from students, faculty, staff, and parents. Worrel (2000) used the Instructional Climate 
Inventory Form S (by Braskamp and Maehr, 1988) to measure students’ perceptions of 
school climate. Hoy, Hoffman, Sabo, and Bliss (1996) used the Organizational Climate 
Descriptive Questionnaire, in both versions for elementary and secondary schools, to 
measure the perceptions of teachers about the climate in secondary and elementary 
schools. Pashiards (2000) also assessed climate at elementary and secondary schools. The 
author used the School Climate Questionnaire to assess teachers and principals’ beliefs 
about climate in their schools. 

Another instrument developed to measure school climate is the Organizational 
Climate Index (OCI), developed by Hoy (2001). It consists of four dimensions: 

a) Collegial Leadership measures whether principal “treats teachers as 
professional colleagues, is open, egalitarian, and friendly, but at the same time 
sets clear teacher expectations and standards for performance.” 

b) Professional Teacher Behavior measures whether teachers show “respect for 
colleague competence, commitment to students, autonomous judgment, and 
mutual cooperation and support.” 

c) Achievement Press measures whether the school ‘set high but achievable 
academic standards and goals; students persist, strive to achieve, and are 
respected by each other and teachers for their academic success; parents, teachers, 
and the principal exert pressure for high standards and school improvement. ’ 
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d) Institutional Vulnerability measures whether the school “is susceptible to a 
few vocal parents and citizen groups. High vulnerability suggests that both 
teachers and principals are unprotected and out on the defensive.” 

Instruments are usually analyzed using classical test theory, in which the item 
statistics depend on sample characteristics. In other words, a mean score on a climate 
instrument reflects the perspectives of the respondents, independently from whether the 
items on that scale were easier or harder to be selected (endorsed). On the other hand, 
Rasch measurement takes into account the ability level of the respondent as well as the 
difficulty of the item (Wright, 1977). In other words, a logits score of teachers’ 
perception of school climate obtained by using a Rasch procedure will reflect how strong 
the individual’s attitudes, opinions, or perspectives are, independently from how hard or 
easy the items were to be endorsed. 

Purpose of the study 

The purpose of this study is to evaluate a modified Organizational Climate Index 
(OCI) using Rasch measurement. 

Methods 

A modified Organizational Climate Index (OCI) was used to assess the overall 
climate in four schools at an urban and a rural district in Ohio. Of the 30 original items 
(including three filler items), 22 were used in this study. 

110 teachers completed a questionnaire, containing OCI, among other scales. This 
questionnaire was eight-pages long. It was reviewed by a panel of experts, prior to being 
administered. They were filled out at staff meetings, a procedure that guaranteed 
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response rate per school ranging from 66.7% to 87.9%. This questionnaire was part of the 
evaluation of an intervention program implemented at schools in academic risk. 

Data were analyzed using WINSTEPS, a Rasch-model computer program 
(Linacre, 2000). 

Results 

The WINSTEPS analysis yielded the simultaneous calibration of items and 
persons. It also provided Separation Index, Reliability, Chi-square statistics, and Fit 
statistics. The person and item variable map is a good place to start with because it helps 
to understand if the calibration process yielded useful information. 

Insert Figure 1 



The range item measures was from approximately +1.4 logits to -1.4 logits, while 
the person measures showed a normal distribution with a wide range from +2.0 logits to - 
2.0 logits. The items that were difficult to endorse were “Parents exert pressure to 
maintain high standards” and “Parents press for school improvement.” On the other hand, 
items such as “Teachers help and support each other,” “Teachers respect the professional 
competence of their colleagues,” and “The interactions between faculty members were 
cooperative” appeared to be the easiest items to endorse. 

The summary of the person and item statistics also provides valuable information 
about the calibration of items and teachers. For items, the separation reliability was very 
high (.96); and for teachers, the reliability was reported as .83. Comparing the observed 
standard deviation of the person abilities (.73) with the adjusted standard deviation, 
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which represents the standard deviation of the person abilities when the measurement 
error is removed, indicated that most of the observed variance is in fact due to the actual 
differences in the teacher’s perception of school climate rather than measurement error. 



Insert Table 1 

Fit statistics are presented by two measures for persons and items. The Outfit 
statistics are unweighted mean square residuals that are specifically sensitive to the 
outliers, whereas the Infit statistics are more sensitive to unexpected responses near the 
point where decisions are made (Linacre, 1994). For the purpose of this study, the upper 
limit of 1 .8 and lower limit of 0.5 were adopted to identify the misfitting items and 
persons. Item fit statistics identified two noisy items out of 22 items, which were “The 
principal responds to pressure from parents” and “A few vocal parents can change school 
policy.” The reason may be due to the fact that teachers do not have much experience 
about those incidents. When infit and outfit statistics were examined; only 7.2 percent of 
the persons have noisy measures. 



Insert Table 2 



Insert Table 3 
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Discussion 

The analysis indicated that all items were working well, with the exception of the 
items “The principal responds to pressure from parents” and “A few vocal parents can 
change school policy.” There were more variations in those items than expected. This 
might be due to the fact that community relations in the sample schools are not so 
prominent. This information was obtained from other aspects of the questionnaire as well 
as other components of the overall evaluation of intervention program of which the 
questionnaire is a part. This may also help explain why the items “Parents exert pressure 
to maintain high standards” and “Parents press for school improvement” were the most 
difficult to endorse. 

Overall, according to the Rasch measurements analysis conducted in the study, 
the modified Organizational Climate Index is a reliable instrument to assess school 
climate. Noisy items could be further explored by conducting analyses at the building 
level as recommended by Hoy, Smith, and Sweetland (2002). In this study, the unit of 
analysis was teachers. Such items could also be explored by comparing schools with 
different levels of community relations because it appeared that the schools in this sample 
did not present strong community relations. 
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Figure 1 

Variable Map of Teacher Measures , and Item Calibrations 
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Table 1 

Summary Statistics 



SUMMARY OF 110 MEASURED teachers 

+ + 

I RAW MODEL INFIT OUTFIT I 

I SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD I 



MEAN 58.2 21.9 .34 .30 .99 -.3 1.00 -.3 
S.D. 9.5 .7 .80 .02 .51 1.7 .53 1.7 
MAX. 78.0 22.0 2.18 .37 2.81 4.4 3.01 4.6 
MIN. 36.0 15.0 -1.53 .28 .24 -4.3 .26 -4.0 



I REAL RMSE .32 ADJ.SD .73 SEPARATION 2.24 teache RELIABILITY .83 | 

I MODEL RMSE .30 ADJ.SD .74 SEPARATION 2.49 teache RELIABILITY .86 I 

I S.E. OF teacher MEAN = .08 | 

+ + 

SUMMARY OF 22 MEASURED items 

+ + 

I RAW MODEL INFIT OUTFIT I 

I SCORE COUNT MEASURE ERROR MNSQ ZSTD MNSQ ZSTD I 



MEAN 291.2 109.5 .00 .13 .98 -.4 1.00 -.3 
S.D. 43.1 1.3 .73 .00 .33 2.6 .35 2.6 
MAX. 353.0 110.0 1.37 .14 1.80 5.4 1.88 5.8 
MIN. 213.0 104.0 -1.04 .13 .54 -4.5 .56 -4.3 



I REAL RMSE .14 ADJ.SD .71 SEPARATION 5.10 item RELIABILITY .96 I 

I MODEL RMSE .13 ADJ.SD .72 SEPARATION 5.41 item RELIABILITY .97 | 

I S.E. OF item MEAN = .16 I 

+ + 
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Table 2 

Sunimary Statistics for Items 
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Table 3 

Sunimary Statistics for Teacher Measures 
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